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© Disclosed is an automatic travelling apparatus which is capable of taking an image of an area ahead of a 
vehicle by an image pick-up device attached to the vehicle; determining a permissible travelling area in the X-Y 
coordinates havmg the X-axis corresponding to the vehicle's running direction on the basis of dot unit data 
sampled from the taken image; setting a target course by a sequence of dots in the permissible travelling area- 
setting a target point on the target course at a specified distance in the X-axis direction and steering the vehicle 
so as to bring rt to the target point, and also characterized by the possibility to convert the discontinuous dot 
pattern of target course into a continuous curve and to accurately set thereon the target point. 
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BACKGROUND OF THE INVENTION 



The present invention relates to an automatic travelling apparatus capable of finding out a permissible 
travelling area for the purpose of permitting a vehicle to automatically run on a road. 
5 Recently, there has been developed such an automatic travelling apparatus which is capable of finding 
out a permissible travelling area, setting a target course thereon and steering a vehicle so as to follow the 
target course. 

The automatic travelling apparatus disclosed in the Japanese laid open patent publication No. 199610- 
88 is intended to take an image of an area ahead of a vehicle in its running direction by an image pick-up 

ro device attached to the vehicle; extract therefrom sequences of points, e.g. discontinuously dotted lines of 
road edges and the like, on the basis of dot image data obtained taken by data sampling of the image; 
determine a permissible travelling area by the X-Y coordinates in the vehicle's running direction on the 
basis of the extracted sequences of points; set a target course in the determined permissible travelling area; 
set a target point on the target course at a specified distance in X-axis direction, which calculated by 

15 multiplying an current running speed by a specified duration; a estimate a steering amount necessary for 
permitting the vehicle to attain the target point on the basis of the currently detected running condition such 
as steering angle, yaw rate (a change of. angular velocity in yaw direction) of the running vehicle; and 
perform steering control of the vehicle to follow the target course with reference to the steering amount in a 
specified control cycle. 

20 Since the above-mentioned automatic travelling apparatus sets a target point on a target course 
represented by a discontinuously dotted line, i.e. a sequence of points, by specifying a given distance in 
the X-axis direction, the target point thus determined may have a decreased accuracy, thereby causing a 
decrease in accuracy of steering control to follow the target course. 

25 SUMMARY OF THE INVENTION 

In view of the foregoing, the present invention was made to provide an automatic travelling apparatus 
which is capable of converting a target course indicated in the form of a sequence of points into a 
continuous curve, and accurately setting a target point on the target course thus continuously represented 
30 to assure an increased accuracy of steering control of the vehicle to follow the target course. 

BRIEF DESCRIPTION OF DRAWINGS 

Fig. 1 is a block diagram showing a structure of an automatic travelling apparatus embodying the 
35 present invention. 

Fig. 2 shows line segments of a road edges obtained by data processing of an image taken by a video 
camera. 

Fig. 3 shows an image obtained by projective transformation of the image shown in Fig. 2. 
Fig. 4 shows an example of a target course set in a permissible travelling area between detected road 
40 edges. 

Figs. 5 (a) and 5 (b) show target courses to be set on a road, which a vehicle is to follow at a low speed 
(a) and a high speed (b) respectively, 

Rg. 6 shows a relationship between a target course and a presumed course. 
Fig. 7 shows a relationship between a vehicle's steering angle and its turning radius. 
45 Fig. 8 shows a sequence of dots representing a target course. 

Fig. 9 shows a target point being set on a target course indicated by a curve. 
Rg. 10 shows a line segment in the X-Y coordinates. 

Fig. 11 shows a point in the p-9 point coordinates obtained by the Hough conversion of the line 
segment shown in Fig, 1 1 . 
so Rg. 12 is a control flow chart according to the present invention. 

In the drawings, 1 is an image pick-up device, 2 is an image processor, 3 is a means for determining a 
permissible travelling area, 4 is a means for setting a target course, 5 is a control means, 6 is a running 
speed sensor, 7 is a yaw rate sensor, 8 is a steering angle sensor, 9 a steering control, 10 a steering drive, 
11 is a vehicle. RA is a permissible travelling area, OC is a target courses indicated by a sequence of 
55 points, OC is a target course indicated by a curve, P is a position of a vehicle and O is a target point 
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Referring now to the drawings, a. preferred embodiment of the present invention will be described in 
detail as follows: 

It will be observed from Fig. 1 that an automatic travelling apparatus according to the present invention 
comprises: an image pick-up device 1 such as a video camera attached to a vehicle for continuously 

5 picking up subsequent images of ground ahead of the vehicle; means 2 for sampling data from images 
taken by the image pick-up device 1 and processing data of the dot image to extract therefrom sequences 
of points spaced from each other such as road edges and the like; means 3 for determining, on the basis of 
the obtained sequences of points, a permissible travelling area such as a road in the direction in which the 
vehicle is to travel; means 4 for setting a target course in the permissible travelling area thus determined; 

w means 5 for determining the instantaneous running condition of the vehicle on the basis of an output signal 
from a speed sensor 6 representing the vehicle's running speed V, an output signal from a yaw rate 
sensor 7 representing the yaw rate V (a change of angular velocity in the yaw direction) and an output 
signal from a steering angle sensor 8 representing the steering angle "8", and for estimating, on the basis 
of the instantaneous running condition, a steering amount for permitting the vehicle to follow the target 

is course; and means 9 (and a steering drive 10) for steering the vehicle with reference to the steering 
amount. 

Actually, a microcomputer aided control is used in place of means 2, 3, 4 and 5, and means 9 can be 
included in the microcomputer aided control, if occasions demand. 

The extraction of sequences of points such as road edges from the taken image in the image 
20 processing means 2 can be made as follows: 

First, each. image supplied from the image pick-up device 1 is sampled in form of dot image input 
signal which is subjected to differentiation process for detection of the road edges. Then, an automatic 
threshold setting circuit in the image processing means 2 sets an optimum threshold value In consideration 
of the degree of shade of the road edge image information just processed. The road edge image will be 
25 subjected to binary transformation. 

Alternately, first, the image may be subjected to binary transformation, and then the binary data may be 
subjected to differentiation. In place of binary transformation poly-digitization may be performed to express 
some shade details of the image. 

Digitized image information will be subjected to the Hough conversion to convert the X-Y linear 
so coordinates into the corresponding p-e point coordinates, thus jointing sequential points and eliminating 
isolated points to provide sequences of points representing the road edges as shown in Fig. 2. 

$ stands for an angle formed between the X-axis and a perpendicular from the origin of the X-Y 
coordinates to a line segment, whereas P stands for the length of the normal line. For instance, the line 
segment I in the X-Y coordinates in Fig. 10 is expressed as the point 01 in the p-fl point coordinates in Fig. 
35 11. 

On that occasion, edge tracing may be performed on the basis of binary-coded image information to 
obtain road edges represented by sequences of points having continuity. The Hough conversion, edge 
tracing and other appropriate processings may be performed simultaneously, and then synthetic judgement 
may be made on the results of these processings to obtain a precise road edge information. 
40 Since the image taken by a video camera of the image pick-up device 1 represents a perspective view, 
the permissible travelling area determining means 3 converts the perspective road edge image as shown in 
Fig. 2 into a non-perspective road edge image as shown in Fig. 3 according to the known projective 
conversion process. 

The permissible travelling area determining means 3 has a projective conversion characteristics set in 
45 consideration of the perspective characteristics of associated video cameras. 

The permissible travelling area determining means 3 can determine, on the basis of the non-perspective 
road image obtained by projective conversion, for instance, an area between the continuous road edges E1 
and E2 shown in Fig. 4 as a permissible travelling area RA in the X-Y coordinates wherein the X-axis 
corresponds to the direction in which the image is taken by the image pick-up device 1, i.e., the vehicle 
50 travels. 

In Fig. 4, a instantaneous position of the vehicle 11 is indicated at a point "P", and the video camera of 
the image pick-up means 1* is mounted at a predetermined position on the vehicle whereat the point "P" 
may appear at the center lower point of the display screen as the origin of the X-Y coordinates. 

After a permissible travelling area is determined by the permissible travelling area determining means 
55 3, the target course setting means 4 will select a course most appropriate for running in the permissible 
travelling area, and will set the so selected course as a target course to follow. 

Preferably, the course may be determined in consideration of the road contour and the running speed 
of the vehicle to meet the instantaneous travelling condition of the vehicle. However,- the course may be 
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basically determined in consideration of the width of the road as follows: 

In case that the target course setting means 4 finds that the width of the road is above a predetermined 
extent, and that vehicles must keep to the left, a target course OC will be set a given constant distance "w" 
(for instance, 1 .5 meters) apart from the left edge of the road, as shown in Fig, 4, 
s In case that the width of the road is below the predetermined extent, a target course will be set along 
the center line (not shown) of the road. 

The coordinates of every sequential points representing the target course In the X-Y coordinate system 
are stored in an internal memory of the target course setting means 4. 

The divisions of the X-Y coordinates for the permissible travelling area and the target course are 
w selected In compliance with the magnification of the video camera of the image pick-up device 1 . 

In Rg. 4, the trace of the vehicle from n P° to "0" represents the course actually followed by the vehicle 
under the control of the control means 5 until the vehicle has come to the target course OC at the point 0. 

According to the present invention, it is also possible to set the most desirable target course in 
consideration of the running condition of the vehicle as follows: 
is In case that the target course setting means 4 finds that the running speed measured by the speed 
sensor 6 is below a predetermined speed, the target course OC will be set in conformity with the road 
contour as seen from Rg. 4. 

When the running speed of the vehicle is higher than a predetermined speed, and when the vehicle is 
running a curved road as shown in Rg. 5(b), a target course OC of reduced curvature is set so as to reduce 
20 the lateral force which is applied to the vehicle. 

After setting a target course in the road, the control means 5 will estimate a steering amount to permit 
the vehicle to follow the target course as follows: 

In the control means, a course which the vehicle will run along is presumed on the basis of the 
currently detected running condition, a deviation of the presumed running course from the target course is 
25 calculated, a correction value of steering angle for permitting the vehicle to follow the target course is 
determined, and then a target steering amount is estimated therefrom. 

Practically, for example, a point where the vehicle will attain in the predetermined time at the currently 
detected running speed is presumed on the presumed course and the target point where the vehicle has to 
attain is set on the target course at the distance calculated by multiplying the presumed duration by the 
30 running speed, and then lateral deviation of the presumed point from the target point is measured to finally 
determine thereby a corresponding correction amount of steering angle. 

Referring now to Rg. 6, it is assumed that a vehicle 11 at Point "P tt is to be brought to the target 
course OC. 

Rrst, the distance L (m) (L= v x T) on the X-axis which the vehicle 11 can run in T seconds will be 
35 determined on the basis of the vehicle's running speed v (m/s) which is detected by the speed sensor 6. 
Then, Point N is determined in the X-axis direction at the distance L apart from Point P (i.e. from the Y-axis) 
whereas a target point 0 is set on the target course OC at the same perpendicular distance from the Y-axis 
as point N has. The lateral deviation y I from Point N to Point 0 (on which the vehicle would be in Tm 
seconds if it travelled straight along the X-axis) is estimated. 
40 Second, in like manner, the course AC which the vehicle 11 is supposed to follow will be estimated 
from the yaw rate T (rad/sec), and point M on the presumed course AC at the same distance from the Y- 
axis as point N has. Then the lateral deviation y m from Point N to Point M (on which the vehicle would be 
in Tm seconds if it travelled straight along the X-axis) is estimated as follows: 
Let a radius of the presumed course be represented by R: 

45 

y m 

- R - {R 2 - (v X tm) 2 } m 
= R-R{1 -{1 - (L/R) 2 } 1/2 

so When R » L, we obtain 

y m 

= R - R {1 - (L/R) 2 12} 
= L 2 /2R (1) 

55 

Since 

r = v/R (2) 
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From Equations (1) and (2) 
y m = L 2 7 / 2v (3) 

5 

The positive sign of yaw rate T represents that the presumed course AC turns to the left whereas the 
negative sign indicates the presumed course turning to the right. 

The yaw rate At to which the yaw rate of the vehicle is to be corrected on the basis of the difference 
w "e" between two deviations y i and y m will be determined from the following equation; 

At = e X 2v / L 2 (4) 



75 Then, on the basis of the steering angle B detected at Point P by the steering angle sensor 8, the 
steering amount 5* for permitting the vehicle to get on the target course OC can be determined as follows: 
Referring to Fig. 7, when R » L, the following equation can be obtained: 

a = 1 1 r (5) 

20 

From equations (2) and (5) we can derive 
5 « ( l I v ) t (6) 

25 where i stands for a wheelbase. According to the equation (6) the steering angle AS for correction in 
accordance with the yaw rate At to be corrected can be given by the following equation: 

AS = ( l / V ) At (7) 

30 In consideration of a general equation of steering angle in relation to the running speed, i.e., substituting 
i =(1 + Kv 2 ) into the equation (7), we can obtain 

AS * At { t (1 + Kv 2 ) / v } (8) 

35 where "K" is a constant which is determined both from the the vehicle characteristics such as tyre 
characteristics, wheelbase and so on. 

Therefore, a steering amount $' for permitting the vehicle to get on the target course can be obtained 
by the following equation: 

40 S x « 5 + AS (9) 



In response to the steering amount 6' given from the controlmeans 5 the steering control 9 issues a 
drive command to the steering drive 10 which in turn steers the vehicle toward the target course OC, 
45 The above-mentioned processing operations will be repeated at an interval of a specified control cycle, 
thereby the steering control of the vehicle may be continuously performed to permit the vehicle to follow 
the target course OC successively resetable in the permissible travelling area which is successively 
renewable at every control cycle as the vehicle travels. 

The automatic travelling apparatus according to the present invention further includes means for 
so converting a sequence of spaced points representing a target course OC into a continuous curve whereon a 
target point 0 will be set. 

The above-mentioned means is practically included In the control means 5, whereby a approximate 
functional curve is calculated from the dot data of a target course OC by the least square approximation. 

It Is well known that a sequence of points can also be approximated to a curve by other general 
55 methods such as the Fourier transformation, Lagrange's interpolation, Spline approximation and so on. 

The calculation of functions of a curve being approximate to sequential dots of a target course by the 
least square method is described in detail as follows: 

As shown in Fig. 8, a function f = y(x) of the curve which is approximated by the least square method to 
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the target course OC represented by a sequence of points in X-Y coordinates may be given by the 
following equation: 

y (x) = a m x n + a m .i x"* 1 + ... + a, x + a 0 ... (10) 



In respect to all dot data (xi , fi),(xj> ,f 2 ) ... (x n ,f n ) of the target course OC the following supplementary 
matrix C is formed: 
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(11) 



20 A coefficient matrix A = C* * C is calculated, 
where C l is a transpose of C. 
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Finally, linear equations of m + 1 ( A * a = b) are obtained and then coefficients aj (j=0 t 1, 2,..., n) of 
the function y(x) indicated iri the equation (10) are calculated thereby. 

By applying thus obtained function y(x) the target point 0 can be set on the target course OC in the 
following way: 

55 For example, as shown in Rg. 9, when XI stands for a value of distance L determined by multiplying 
the running speed v by the predetermined duration tm, said value XI substituted into the function y(x) to 
determine a value yi t by which the target point 0 is specified on the target course OC* represented by an 
approximated continuous curve, 
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Fig. 12 shows a flow of contra! in the apparatus according to the present invention. 

Namely, according to the present invention, it is possible to, in place of directly setting a target point O 
on a discontinuousiy dotted target course OC, locate the target point by use of a function y(x) representing 
the target course OC by a continuous curve, thereby assuring higher accuracy of target setting in respect to 
5 the target course. 

As be apparent from the foregoing description, the automatic travelling apparatus according to the 
present invention offers such an advantage that when taking an image of ground area ahead of a running 
vehicle by an image pick-up device attached to said vehicle; determining a permissible travelling area in the 
X-Y coordinates having the X-axis corresponding to the vehicle's running direction on the basis of dot unit 

70 data sampled from the taken image; setting a target course by a sequence of points in the permissible 
travelling area; setting a target point on the target course at a specified distance in the running direction of 
the vehicle and steering the vehicle so as to bring it to the target point, it is particularly capable of 
converting the discontinuousiy dotted target course into a continuous curve and accurately setting thereon 
the target point, thereby assuring higher accuracy and stability of steering control for permitting the vehicle 

75 to follow the target course. 

The invention as described above can be summarized as follows: 

Disclosed is an automatic travelling apparatus which is capable of taking an image of an area ahead of 
a vehicle by an image pick-up device attached to the vehicle; determining a permissible travelling area in 
the X-Y coordinates having the X-axis corresponding to the vehicle's running direction on the basis of dot 
20 unit data sampled from the taken image; setting a target course by a sequence of dots in the permissible 
travelling area; setting a target point on the target course at a specified distance in the X-axis direction and 
steering the vehicle so as to bring it to the target point, and also characterized by the possibility to convert 
the discontinuous dot pattern of target course into a continuous curve and to accurately set thereon the 
target point 

25 

Claims 

1, An automatic travelling apparatus comprising: 

means for taking an image of an area ahead of a running vehicle by an image pick-up device attached 
30 to the vehicle; 

means for sampling data from the image taken by the image pick-up means; 

means for determining, on the basis of the sampled image data, a permissible travelling area ahead of 
the vehicle in X-Y coordinates, X-axis thereof corresponding to the vehicle's running direction; 
means for setting a target course by a sequence of points in the permissible travelling area; 
55 means for setting a target point on the target course at a specified distance ahead of the vehicle in the 
X-axis direction, 

control means for steering the vehicle to bring it to the target point 

characterized by converting the target course from a sequence of dots into a continuous curve and 
setting the target point on said curve. 

40 

2. Each and every novel feature or novel combination of features disclosed in the specification and the 
drawing. 
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FIG. 12 
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